It is known from EP-A-495 622 to use a camera-projector arrangement positioned above a desk, in order to enable a user to select functions to be performed by selecting items located within the field of view of the camera. A video camera or scanner is used to capture images of documents on a desk, and feedback information is displayed by means of a projection display. The functions include calculating and translating operations carried out on data (e.g., in a paper document) located on the desk. In such a system, camera and display must be kept in registration, and this involves calibration whenever the camera's field of view changes with respect to the display.
By way of example, when the user selects a word in a document, feedback must be displayed to confirm the selection, aligned precisely with the word. The position of the displayed information can be calculated if the size, orientation and position of the camera's field of view are known, relative to the display coordinate system.
Previous proposals for solving this calibration problem have used calibration marks projected onto the work surface. EP-A-622,722 discloses a calibration system which projects a thick cross or "plus" sign (+), and uses image morphology (see D. Bloomberg & P. Maragos, "Image Algebra and Morphological Image Processing", SPIE Conference Procs, San Diego, Calif., July 1990) to pinpoint the center of the mark in the frame grabber coordinate space. A cross (+) is projected at four successive points; and to calculate the mapping from the four points the techniques uses the following equations (see FIG. 4 of EP-A-622,722), EQU x'=c.sub.1 x+c.sub.2 y+c.sub.3 xy+c.sub.4 EQU y'=c.sub.5 x+c.sub.6 y+c.sub.7 xy+c.sub.8
where (x,y) are coordinates in the projected display, and (x',y') are coordinates in the frame grabber. With four point pairs, the set of simultaneous linear equations can be solved by Gaussian Elimination. Then, a fifth cross (+) is projected and its location is checked to make sure it is close enough to the position produced by the mapping. While this process deals with the keystoning and rotation effects caused by lack of alignment of the camera to the display, and the result is accurate to within one or two display pixels, it requires the user manually to specify the rough position of the camera's field of view, and is thus inappropriate for situations where the field of view is constantly or frequently changing.
There is therefore a need for a calibration technique which can be performed rapidly and automatically whenever the field of view changes. There is also a need for a calibration technique which is effective even when the work surface is cluttered with documents lying within the field of view of the camera. And there is also a need for a calibration technique which can cope with a wide range of fields of view.